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Abstract 

A general expression for the distortion rate function (DRF) of cyclostationary Gaussian processes in terms of 
their spectral properties is derived. This expression can be seen as the result of orthogonalization over the different 
components in the polyphase decomposition of the process. We use this expression to derive, in a closed form, the 
DRF of several cyclostationary processes arising in practice. We first consider the DRF of a combined sampling and 
source coding problem. It is known that the optimal coding strategy for this problem involves source coding applied 
to a signal with the same structure as one resulting from pulse amplitude modulation (PAM). Since a PAM-modulated 
signal is cyclostationary, our DRF expression can be used to solve for the minimal distortion in the combined sampling 
and source coding problem. We also analyze in more detail the DRF of a source with the same structure as a PAM- 
modulated signal, and show that it is obtained by reverse waterfilling over an expression that depends on the energy 
of the pulse and the baseband process modulated to obtain the PAM signal. This result is then used to study the 
information content of a PAM-modulated signal as a function of its symbol time relative to the bandwidth of the 
underlying baseband process. In addition, we also study the DRF of sources with an amplitude-modulation structure, 
and show that the DRF of a narrow-band Gaussian stationary process modulated by either a deterministic or a random 
phase sine-wave equals the DRF of the baseband process. 


I. INTRODUCTION 

The distortion rate function (DRF) describes the average minimal distortion achievable in sending an information 
source over a rate-limited noiseless link. Sources with memory posses an inherent statistical dependency that can 
be exploited in the context of data compression. However, not many closed-form expressions for the DRF of such 
sources are known, and those are usually limited to the class of stationary processes. Two notable exceptions are 
the DRFs of the Wiener process, derived by Berger m, and of auto-regressive Gaussian processes, derived by 
Gray El. Indeed, information sources are rarely stationary in practice, and source coding techniques that are based 
on stationary assumptions about the source will likely achieve poor performance if the source has time-varying 
statistics. 

Cyclostationary processes (CS) (also known as periodically correlated processes or block-stationary processes) 
are a class of non-stationary processes whose statistics are invariant to time shifts by integer multiples of a given 
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time constant, denoted as the period of the process. As described in the survey by Gardner la, CS processes 
have been used in many fields to model periodic time-variant phenomena. In particular, they arise naturally in 
synchronous communication where block coding and modulation by periodic signals are used. Spectral properties 
of CS processes, which will be used in our derivations, are also reviewed in a and in the references therein. 

In this paper we analyze the DRF of CS Gaussian processes. Our study of the DRF of CS processes can be 
motivated by a very simple example: we are interested to find the DRF of the process obtained by modulating a 
continuous-time Gaussian stationary process U{-) by a cosine wave with random phase <I>, namely 

X^{t) = VlU{t) cos {Infot+ ^), f G R, (1) 

where <I> is uniformly distributed over [Q,27t). This process is commonly given as an example of a wide-sense 
stationary process in signal processing textbooks (e.g. m Ex. 8.18]). Note that due to the random phase, i^<t>(-) is 
not Gaussian and in fact is non-ergodic. It seems that in the context of rate-distortion theory, the spectrum of X(j)( ) 
can only be used to derive an upper bound on its DRF given by the DRF of a Gaussian stationary process with the 
same second order statistics 0 Thm. 4.6.5]. The theory of asymptotic mean stationary (AMS) processes @ implies 
that the DRF of X^{t) is given by the DRF of each one of its ergodic components ||2l Thm. 11.3.1], corresponding 
to different values of the phase (p G [0,27r). Each ergodic component satisfies a source coding theorem which allows 
us to evaluate its DRF using an optimization over probability distributions subject to a mutual information rate 
constraint. One might think this decomposition might provide us with a recipe to evaluate the DRF of ) by 
averaging over the DRF of the process X,p(-), obtained by fixing the phase in X(j)( ). However, while the process 
X,p(-) is Gaussian, it is no longer stationary - but rather CS. While an expression for the DRF of CS processes 
is known la, Q, it seems that the only existing mechanism for evaluating this DRF is by the Karhunen-Foeve 
(KF) expansion 0. In this method it is required to solve for the eigenvalues of a Fredholm integral equation for 
each finite blocklength, and use a waterfilling expression over these eigenvalues. The DRF is finally obtained in 
the limit as the size of the blocklength goes to infinity. This evaluation, however, does not exploit the special block 
periodicity of the CS source. Moreover, it does not provide any intuition on the optimal source coding technique 
in terms of spectral properties of the source. In contrast, the DRF for a stationary Gaussian process is obtained by 
waterfilling over its power spectral density, which provides clear intuition about how the source code represents 
each frequency component of the signal Qni. 

In this work we derive an expression for the DRF of Gaussian CS processes which uses their spectral properties, 
and therefore generalizes the waterfilling expression for the DRF of Gaussian stationary processes derived by Pinsker 
ED. This expression is obtained by considering the polyphase components of the process, which can be seen as a 
set of stationary processes that comprise the CS process ifTSIl . We show that the DRF of a discrete-time CS can be 
obtained in a closed form by orthogonalizing over these components at each frequency band. For continuous-time 
CS processes, we obtain an expression which is based on increasingly fine discrete-time approximations of the 
continuous-time signal. The DRF evaluated for these approximations converges to the DRF of the continuous-time 
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process under mild conditions on its covariance function. 

The main results of this paper are divided into two parts. In the first part we derive a general expression for 
evaluating the DRF of a second order Gaussian CS process in terms of its spectral properties. This expression 
is given in the form of a reverse waterfilling solution over the eigenvalues of a spectral density matrix defined 
in terms of the time-varying spectral density of the source. For discrete-time Gaussian processes, the size of this 
matrix equals the discrete period of the source. We extend our result to Gaussian CS processes in continuous-time 
by taking increasingly finer discrete time approximations. The resulting expression is a function of the eigenvalues 
of an infinite matrix. In addition, we derive a lower bound on the DRF which can be obtained without evaluation 
of the matrix eigenvalues. We show that this bound is tight when the polyphase components of the process are 
highly correlated. 

In the second part of the paper we use our general DRF expression to study the distortion-rate performance in 
more specific cases. Specifically; 

• We derive a closed form expression for the DRF of a process with a pulse-amplitude modulation (PAM) signal 
structure. We show how this expression can be used to derive the minimal distortion in estimating a stationary 
Gaussian process from a rate-limited version of its sub-Nyquist samples. 

• We study the effect of the symbol time in PAM on the information content of the modulated signal at the 
output of the modulator. 

• We evaluate in a closed form the DRF of a Gaussian stationary narrowband process modulated by a deterministic 
cosine wave. We show that the DRF of the modulated process equals that of the baseband stationary Gaussian 
process provided the latter is narrowband. We further conclude that the stationary, non-Gaussian and non-egodic 
process given by ([T]i above has DRF identical to the DRF of the modulated process without the random phase. 
These two results imply that the DRF of the stationary non-Gaussian amplitude modulated process is strictly 
smaller than the DRF of a Gaussian stationary process with the same second order statistics. 

The rest of this paper is organized as follows: in Section |II] we review concepts and notation from the theory of 
CS processes and rate distortion theory. Our main results are given in Section lUD where we derive an expression 
for the DRF of a Gaussian CS process. In Section |IV] we derive a lower bound on this DRF. In Section |V] we 
explore applications of out main result in various special cases. Concluding remarks are provided in Section IVll 

II. Definitions and Notations 

A. Cyclostationary Processes 

Throughout the paper, we consider zero mean Gaussian processes in both discrete and continuous time. We use 
round brackets to denote a continuous time index and square brackets for a discrete time index, i.e. 

x(.) = {x(f),f gR}, 

and 

X[\ = {X[n\,nC'L}. 
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Matrices and vectors are denoted by bold letters. 


The statistics of a zero mean Gaussian process X() is specified in terms of its autocorrelation functiorQ 

Rx{t,z)=E[X{t + z)X{t)]. 


If in addition the autocorrelation function is periodic in t with a fundamental period Tq, 


Rx{t + To,z) =Rx{t,'c), 

then we say that X(-) is a cyclostationary process or simply cyclostationt^ ja, m- We also assume that Rx{t,T) 
is bounded and Riemann integrable on [0, Tq] x R, and therefore 

= lim f EX{tfdt = ^ [ Rx{t,0)dt 

T^oo 11 1 q Jq 

is finite. 

Suppose that Rx{t,T) has a convergent Fourier series representation in t for almost any T S R. Then the statistics 
of X(-) is uniquely determined by the cyclic autocorrelation (CA) function; 

1 /■^o /2 

= T Rx{t,x)e-^^^'"l'^^dt, nez. (2) 

h J-Tq/2 

The Fourier transform of RJ(t) with respect to T is denoted as the cyclic power spectral density (CPSD) function; 

5J(/) = 1“ -oo<f<oo. (3) 

If S^{f) is identically zero for all n 7 ^ 0, then Rx{t,T) = Rx{0,t) for all 0 <t <T and the process X(-) is 
stationary. In such a case Sx{f) = S^{f) is the power spectral density (PSD) function of 2f(-). The time-varying 
power spectral density (TPSD) function lfT4l Sec. 3.3] of 2£'(-) is defined by the Fourier transform of Rx{t,T) with 
respect to T, i.e. 

if) = I” Rx it, T)e-2 ^'/Vt. (4) 

The Fourier series representation implies that 


Sxif) = Ls"xin 


^iTiint/To 


( 5 ) 




Associated with every cyclostationary process 2f(-) with period Tq is a set of stationary discrete time processes 
0 < f < To, defined by 

X‘[n]=XiTon + t), n e Z. ( 6 ) 


> In 03 and in other references, the symmetric auto-correlation function 

Rx{t, T) ^ E [X{t + t/2)X(? - t/2)] ^Rx{t- t/2,t), 

the corresponding CPSD S^if) and TPSD are used. The conversion between S^(f) and the symmetric CPSD is given by S^if) — 

SJ(/-n/(27b)). 

^It is customaiy to distinguish between wide-sense cyclostationarity which relates only to the second order statistics of the process, and 
strict-sense cyclostationarity which relates to the finite order statistics of the process 03 Ch. 10.4]. Both definitions coincide in the Gaussian 
case. 
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These processes are called the polyphase components (PC) of the cyclostationary process ^(•). The cross-correlation 
function of [•] and [•] is given by 

^x'ix'2 [W)^] = IE [X[TQ{n + k) + ti]X[T()n +1'^ 

= Rx iTon + t 2 ,Tok + ti —12) 

= Rx{t2iTQk + ti — t2) ■ (7) 


Since Rx‘ix ‘2 depends only on k, this implies that X‘' [•] and X^^[-\ are jointly stationary. The PSD of X‘[-] is 
given by 


kez 


= Y,Rxit,Tok)e-^^‘‘^\ 
kez 

Using the spectral properties of sampled processes, we can use 
and the CPSD of X( ) as follows; 


( 8 ) 

(l8]l and (|5]l to connect the functions Sx' (e^^"^) 


Sx' 


More generally, for fi,t 2 G [0,7o] we have 


=— E Sx 

Toh ^ 



1 


To 


E E 

keZneZ 


t g2;tmf/7b 

To ) 


kez 


( 9 ) 


1 


To 


kez 




= — E E f ^ e27r;(m^ + a75i(0-i:))^ 

^0 feZmeZ \ To J 

We now turn to briefly describe the discrete-time counterpart of the CA, CPSD, TPSD and the polyphase 
components defined in (|2]l, Q, (HI and (|6]l, respectively. 

A discrete time zero mean Gaussian process X [•] is said to be CS with period M S N if its covariance function 


Rx[n,k] = E [X[n-f A:]X[n]] 


is periodic in k with period M. For m — Q,... ,M, the m* cyclic autocorrelation (CA) function of [•] is defined as 

M-l 

R’^[k] 4 ^ Rx[n,k]e-^^‘""‘^^. 

n=0 

The m* CPSD function is then given by 


kez 

and the discrete TPSD function is 

S\{e^^'^) = Y,Rx[n,k]e-^^‘^K 
kez 
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Finally, we have the discrete time Fourier transform relation 


M-l 


^ni^nmjM 


m=0 


The m-th stationary component 0 <m<M — I of X[-] is defined by 

X'”[n] = X[Mn + m], n £ Z. 

For 0 < m,r,n < M — I and A; £ Z we have 

Rx'nxr [n,k]=E [X"^ [n + k]X''[n]] 

= E [X [Mn +Mk + m]X [Mn + r[ ] 
= Rx [Mn + r,Mk + m — r] 

= Rx [r,Mk + m — r[. 


(10) 


( 11 ) 


Using properties of multi-rate signal processing; 


Sx-x'- = Y. Rx[r,Mk + m-r]e-^^‘’^^ 

kez 


The discrete-time counterpart of (|9]l is then 


( 12 ) 


Sx"'x^ 


M-lM-l 


M 


EEs; 


k=0 n=0 




m—r){(p—k) 


(13) 


The functions Sxx'X'', 0<m,r<M—1 define an M x M matrix Sx{e^^'^) with (m-f l,r-f 1)* entry 
Sx"'X' This matrix completely determines the statistics ofX[•], and can be seen as the PSD matrix associated 

with the stationary vector valued process X^[n] defined by the stationary components of X[-]: 


X"[n] 4 (X°[n],...,x"-‘[n]), n £ Z. (14) 

We denote the autocorrelation matrix of X'^[-] as the PSD-PC matrix. Note that the (r-|- 1)* entry of the 

PSD-PC matrix is given by (HB- 


B. Examples 

We present two important modulation models which result in CS processes. 

Example 1 (amplitude modulation (AM)). Given a Gaussian stationary process U{-) with PSD S[j{f), consider 
the process 

XAM{t) = V2U (f) cos (27r/of -I- (p ), 
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where /o > 0 and (p G [0,27r) are deterministic constants. This process is CS with period Tq= ^ and CPSD l\16\ 
Eq. 41 ] 

' Suif + fo)+Suif-fo), m = 0, 

%(/T/o)e±"'^ m±2, 

0 , otherwise. 


SlM{f) = \{ 


This leads to the TPSD 


Slif) = ^%(/ + /0)(l +e-2{2’^''A'+^)) 


(15) 




Example 2 (pulse-amplitude modulation (PAM)). Consider a Gaussian stationary process U{-) modulated by a 
deterministic signal p{t) as follows: 


XpAAiit) = Y. U{nTo)p{t-nTo). (16) 

nGPt 

This process is CS with period Tq and CPSD M6\ Eq. 49] 

SpAMif) = {f- £) if). « e (17) 

where P{f) is the Fourier transform of p{t) and P*if) is its complex conjugate. If Tq is small enough such that 
the support of Pif) is contained within the interval then Sp^j^if) = 0 for all n f 0, which implies 

that 24>4 m(-) stationary. 


C. The Distortion-Rate Function 

For a fixed T > 0, let Xp be the reduction of X( ) to the interval [—T,T]. Dehne the distortion between two 
waveforms x(-) and y{-) over the interval [—T,T] by 

dTixi-),yi-)) = j^^{x{t)-y{t)f dt. (18) 

We expand Xp by a Karhunen-Loeve (KL) expansion ||9] Ch 9.7] as 

Xp{t)^YXkfkit), -T<t<T, (19) 

k=l 

where {/^} is a set of orthogonal functions over [—7’,?’] satisfying the Fredholm integral equation 

hfkit) = ^ J^^Kxit,s)Ms)ds, tG[-T,T], (20) 

with corresponding eigenvalues {Xp}, and where 

Kx{t,s) ^EXit)X{s) = Rxis,t - s). 

Assuming a similar expansion as ( fT9] l to an arbitrary random waveform Yp, we have 

Edp{Xp,Yp) = ^ r E{X{t)-Y{t))^dt= Y E(X„-F„)2. 

-f — T n=—oo 
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The mutual information between X(-) and y(-) on the interval [—7’,?’] is defined by 

It 4 ^^im/(X^^; , 

where = (X_a?, ... ^X^), Y^^ = (T-at, ... ,Tv) and the X„s and y„s are the coefficients in the KL expansion of 
X(-) and T(-), respectively. 

Denote by the set of joint probability distributions PxS over the waveforms (X(-),X(-)), such that the 
marginal of X(-) agrees with the original distribution, and the average distortion IE^77■ (X(-),X(-)) does not exceed 
D. The rate-distortion function (RDF) of X(-) is defined by 

R{D) = lim Rt{D), 

T -^oo 

where 

Rr(D)=inf/r(X(-);X(-)), 

and the infimum is over the set It is well known that Rip) and Rrip) are non-decreasing convex functions of 
D 0, and therefore continuous in D over any open interval. We define their inverse function as the distortion-rate 
functions D{R) and Dt{R), respectively. We note that by its definition, D{R) is bounded from above by the average 
power of X(-) over a single period: 

/ ¥X^{t)dt =lim^ f Rx{t,Q)dt 
T^oaZl J-T T-^oqZ1 J-T 

= 1 rRx{tP)dt = RUQ). 

Iq Jo 

For Gaussian processes, we have the following parametric representation for Rt(D) or Dt{R) lH Eq. 9.7.41] 

Dr(0)= f;min{0,4} (21a) 

A:=l 

^r(0) = if;iog+(A,/0), (21b) 

^ k=i 

where log+A: = max{logA:,0}. 

In the discrete-time case the DRF is defined in a similar way as in the continuous-time setting described above by 
replacing the continuous-time index in (fTsT l. (fT^ and (l20l) . and by changing integration to summation. Since the KL 
transform preserves norm and mutual information, this definition of the DRF in the discrete-time case is consistent 
with standard developments for the DRF of a discrete-time source with memory as in 0 Ch. 4.5.2]. Note that with 
these definitions, the continuous-time distortion is measured in MSE per time unit while the discrete-time distortion 
is measured in MSE per source symbol. Similarly, in continuous-time, R represents bitrate, i.e., the number of bits 
per time unit. In the discrete-time setting we use the notation R to denote bits per source symbol. 

Since the distribution of a zero-mean Gaussian CS process with period To is determined by its second moment 
Rx{t,T), we observe that such processes are 7o-ergodic and therefore block-ergodic as defined in |[8] Def. 1]. 
It follows that a source coding theorem that associates D{R) with the optimal MSE performance attainable in 
encoding X(-) at rate R is obtained from the main result of 0. Specifically for the discrete-time case, it is shown 
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in @ Exc. 6.3.1] that CS processes belong to the class of asymptotic mean stationary process (AMS) 1^, where a 
source coding theorem for AMS processes can be found in El. 


D. Problem Formulation: Evaluating the DRF 

In the special case in which X( ) is stationary, it is possible to obtain D{R) without explicitly solving the Fredholm 
equation (l20l i or evaluating the KL eigenvalues; in this case, the density of these eigenvalues converges to the PSD 
Sx{f) of X( ). This leads to the celebrated reverse waterfilling expression for the DRF of a stationary Gaussian 
process, originally derived by Pinsker ini: 

m = ^ log+ [5x (/) /0]df. (22a) 

D{d)= r mm{Sxif),d}d<l). (22b) 

J — oo 

The discrete-time version of (l22T i is given by 

R{e) = \ log+ [5x {e^’^‘‘^)/9] d^. (23a) 

D{0)= l^mm{Sx{e^’^“^),e}d(^. (23b) 

Equations (l22l l and (l23T l define the distortion as a function of the rate through a joint dependency on the water level 
parameter 9. 

We note that stationarity is not a necessary condition for the existence of a density function for the eigenvalues 
in the KL expansion. For example, such a density function is known for the Wiener process HI which is a 
non-stationary process. 

The main problem we consider in this paper is the evaluation of D{R) for a general CS Gaussian process. In 
principle, this evaluation can be obtained by computing the KL eigenvalues in (l20l i for each T, using (l2Tli to 
obtain Dt{R) and finally taking the limit as T goes to infinity. For general CS processes, however, an easy way to 
describe the density of the KL eigenvalues is in general unknown. As a result, the evaluation of the DRF directly 
by the KL eigenvalues usually does not lead to a closed-form solution. In the next section we derive an alternative 
representation for the function D{R) which is based on an approximation of the kernel Kx{t,s) used in (l20l i. 

III. Main Results 

In this section we derive our main results with respect to an expression for the DRF of a Gaussian CS which 
does not involve the solution of the Fredholm integral equation (l20l i. 
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Our first observation is that in the discrete-time case, the DRF of a Gaussian CS process can be obtained by an 
expression for the DRP of a vector Gaussian stationary source. This expression is an extension of (l2Tt . which was 
derived in M Eq. (20) and (21)] and is given as follows: 

(0) = ^ L / 1 min(24a) 

™ m=l ■'^1 
1 " /-T 1 

= \ /0] (24b) 

m=l 2 

where Ai Am are the eigenvalues of the PSD matrix Sx at frequency (j). We have the 

following result: 

Theorem 1. Let X]-] be a discrete-time Gaussian cyclostationary process with period M G N. The distortion rate 
function of X\^\ is given by 

1 " /-T 

/ j min {Am (25a) 

m=l4-3 

1 " r\ 

E \ log^ /®] (25b) 

where X\ ^ Xm are ^/ze eigenvalues of the PSD-PC matrix with (m+ l,r+ 1)^^ by 

Sx'nxr {e^^“^) ^, ( 26 ) 

Proof: A full proof can be found in Appendix The idea is to use the polyphase decomposition dTSli and the 
stationary vector valued process X“[-] defined in (fT4ll . The PSD matrix of the process is shown to coincide with 
the PSD-PC matrix of X[-]. The proof shows that the DRF of X[-] coincides with the DRF of X^[-]. The result 
then follows by applying (l24l i to X^[-]. ■ 

Equation (l25l l has the waterhlling interpretation illustrated in Fig. [T] the DRF is obtained by setting a single 
water-level over all eigenvalues of (l26l l. These eigenvalues can be seen as the PSD of M independent processes 
obtained by the orthogonalization of the PC of X[-]. Compared to the limit in the discrete-time version of the KF 
expansion (ED, expression (l25l l exploits the CS structure of the process by using its spectral properties. These 
spectral properties capture information on the entire time-horizon and not only over a hnite blocklength as in the 
KL expansion. 

The following theorem explains how to extend the above evaluation to the continuous-time case. 

Theorem 2. Let Xf) be a Gaussian cyclostationary process with period Tq and correlation function Rx{t,x) 
Lipschitz continuous in its second argument. For a given M C N, denote 

1 " 

Dm{9m) = —Y, \ tnin {Xm ,9 m} d^ (27a) 

1 " /-T 

R{9m) = (27b) 

m=l4-3 


10 



Fig. 1: Waterfilling interpretation of (l25l l for M =3. The blue and the yellow parts are associated with equations 
(I25al) and (I25bl i. respectively. 


where Xi < • • • < Xm are the eigenvalues of the matrix Sx with its (m+ l,r+ 1)** entry given 

by 


^ arTo/M f 0_ f£\ 2m(m-r) 


Ys^ 


V To 


-k 

)~M~ 


- ^ L L 

■'0 keznez 


To 


Then the limit of Dm in M exists and the distortion-rate function ofXf) is given by 


(28) 


D{R)= \im Dm {OMiR)). (29) 

Proof sketch: The proof idea is to use a CS discrete-time process that approximates X(-). This approximation 
becomes tighter as M increases, so that the limit in (l29l l converges to the DRP of the continuous-time process 
coincides. The proof details are given in Appendix iBl 


Discussion 

The expression (l27l l is obtained by taking the limit in (l25T l over the time-period of a discrete-time CS process, 
where the code rate R is appropriately adjusted to bits per time unit. Although (IZTT i only provides the DRP in terms 
of a limit, this limit is associated with the intra-cycle time resolution and not with the time horizon as in (l2Tll . This 
fact allows us to express the DRF in terms of the spectral properties of the process, which captures ‘memory’ in 
the process over the entire time horizon. 

We note that limits of the form (l29l l have been obtained in closed-form using Szego’s Toeplitz distribution 
theorem lfT9l Section 5.2] when the underlying process is stationary and the matrix considered is Toeplitz ll20l . 
0, m or block Toeplitz El\ . Unfortunately, the matrix in (l28T l is not Toeplitz or block Toeplitz so Szego’s 
theorem is not applicable. In the following section we provide a few examples where the limit in (|2^ can be 
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obtained in closed form which lead to a closed form expression for the DRF. 

Expression dZTl) can be seen as the extension to CS of the waterhlling expression (l22l i derived for stationary 
processes. While the latter can be understood as the limiting result of coding over orthogonal frequency bands ifTOll . 
expression dZTl i implies that the DRF for CS processes is the result of two orthogonalization procedures; (1) over 
the PC inside a cycle, which is associated with the eigenvalues decomposition of the PSD-PC, and (2) over different 
frequency bands of the stationary processes resulting from the hrst orthogonalization. 

The decomposition of the process into its stationary PCs can be further exploited to derive a lower bound on the 
DRF, which become tight as these PC become highly correlated. This bound is explored in the next section. 


IV. Fower Bound 


In this section we derive a lower bound on the DRF of a Gaussian CS process. This lower bound is expressed 
only in terms of the PCs of the process and does not require the eigenvalue evaluation of Theorems [T] and |2] The 
basis for this bound is the following proposition, which holds for any source distribution and distortion measure 
(although we will consider here only the quadratic Gaussian case). 

Proposition 1. Let X[-] be a vector valued process of dimension M. The distortion-rate function o/X[-] satisfies 

^ M-l 

Dx{R)>j^Y.DxAR). (30) 

m=0 

Proof: Any rate R code for the process X[-] induces a rate R code on each of the coordinates m = 

1. At each coordinate, this code cannot achieve lower distortion than the optimal rate R code for that 
coordinate. ■ 

Proposition [T] applied to Gaussian CS processes leads to the following result: 


Proposition 2. Let X[-] be a discrete-time Gaussian CS process with period M S N. The distortion rate function 
of X[-] satisfies 

1 fi 

D{R) > - E / . min{5x». ,B^}d(^, (31) 


where for each m — 0,... ,M — I, dm satisfies 

1 /■i 

R{Qm) = 2 y_ I 

Here 

1 / A- \ 

4 Sx'nx-n =J^LSX 

n=0 

is the PSD of the m* PC of X[-]. 


(32) 
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Proof: The claim is a direct application of Proposition [T] to our case of a discrete-time CS process: the summands 
on the RHS of (EB are the individual DRF of the PCs m = 0,... ,M- 1, of X[-] obtained by (|23]l. 


Proposition |2] can be extended to the continuous-time case by approximating the outer integral in (l3Tl l by finite 
sums. This yields the following result: 


Proposition 3. Let Xf) be a continuous-time Gaussian cyclostationary process with period Tq > 0 and correlation 
function Rx{t,x) Lipschitz continuous in its second argument. The distortion rate function ofXf) satisfies 




0 — n 


, 9t > d(pdt, 


(33) 


where for each 0 < t < Tq, 9t satisfies 


m) 





d(j). 


(34) 


Proof: See Appendix ICl 

The bound dSTT i is obtained by averaging the minimal distortion at rate R in describing each one of the PCs of 
X(-). For each such component X‘ [•] there is an associated water level 9t obtained by solving (l34l l for 0,. For R — Q, 
9t is always bigger than the essential supremum of 




To 




0 — n 


so the RHS of (|3T1 ) equals the average over the total power of each one of the PCs of X{-) which are summed to 
(7| = Dx{0). On the other hand, if R —oo then 0, ^ 0 for all t G [0,7b], and again equality holds in OTI) . That is, 
the bound is tight in the two extremes of R = 0 and R °o. 

From a source coding point of view, the bound (l33l l can be understood as if a source code of rate R is applied 
to each of the PCs of (•) individually. On the other hand, the DRF in dZTb is obtained by applying a single rate R 
code to describe all these PC simultaneously. As a result, the bound is tight only when all the PCs are maximally 
correlated, i.e. when a single PC determines the rest of them. A case where the latter hold is shown in the following 
example. 


Example 3 (equality in (|33])). Let Xf) be the PAM signal of Example^where the pulse p{t) is given by 

fl 0<f<7b, 
pit) = \ 

I 0 otherwise. 

The sample path of X{-) has a staircase shape illustrated in Figure^ This process is equivalent to the discrete 
time process U[-\ = {U{nTo),n £ Z} both in information rate and squared norm per period Tq, which is enough 
to conclude that DxiR) = Du(RTo). Indeed, the PCs in this case are maximally correlated, in the sense that a 
realization of X^[-] = {X{nTo), n C Z} determines the value of X^[-\ = {X{{nA)To), n C Z} for all 0 < A < 1. In 
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Fig. 2: An example of a continuous-time PAM process that attains equality in ( l33l l. 



Fig. 3: Combined sampling and source coding system model. 


addition, for all 0 <t <To we have 

Si = E , 

nez V ^0 / 

where the latter is the PSD of the discrete time process t7[-], so (I221 l implies that the RHS of (I331 l is the DRF of 
t7[-]. We therefore conclude that the DRF ofX(-) is given by the RHS of (I331 l. 

V. Applications 

In this section we apply the expression obtained in Theorem |2] to study the distortion-rate performance of a few 
CS processes that arise in practice. 

A. Combined Sampling and Source Coding 

We begin with the distortion-rate performance in the combined sampling and source coding problem considered 
in US). This problem is described by the system of Figure |2 the source Uf) is a Gaussian stationary process with 
a known PSD Su{f)- The source is uniformly sampled at rate fs = Tf^, resulting in the discrete time process U[-] 
defined by t7[n] =U {n/ff). The process t7[-] is then encoded at rate R bits per time unit. The goal is to estimate 
the source U{-) from its sampled and encoded version under a quadratic distortion. We denote by the function 
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Du\i){fs:R) the minimal distortion attainable in this estimation, where the minimization is over all collections of 
encoders and decoders operating at bitrate R. Note that if t/(-) is sampled above its Nyquist rate, then there is no 
loss of information in the sampling operation, and we get 

where Du{R) is found by d22li . Therefore, the case of most interest is that of sub-Nyquist sampling of U{-). In 
what follows we use Theorem|2]to derive D^jj{fs,R) in closed form. 


Our first observation is that the combined sampling and source coding problem of Fig. [3 can be seen as an 
indirect source coding problem ||23|: the distortion is measured with respect to the process t/( ), but a different 
process, namely t7[-], is available to the encoder. Wolf and Ziv ll24ll have shown that the optimal source coding 
scheme under quadratic distortion for this class of problems is obtained as follows; the encoder hrst obtains the 
minimal mean square error (MMSE) estimate of the unseen source, and then an optimal source code is applied to 
describe this estimated sequence to the decoder. In the setting of Figure 12 this implies that is attained 

by hrst obtaining the MMSE estimate 

f7(f)=E[t/(f)|t7[.]] 

at the encoder, and then solving a standard source coding problem with the process U{-) as the process to which 
the source code is applied. Moreover, this scheme implies that the distortion decomposes into 2 parts: 


Du\fj{f,,R)=mm^t{U\U)+DQ{R), 


(35) 


where mmse(t/|t/) is the MMSE in estimating U{-) from t/[-], and D^{R) is the DRE of the process U{-). 
Standard linear estimation techniques ll25l leads to 


U{t) = U[n]w{t -riTo) = Y U{nTo)w{t - nTo), 


where the Eourier transform of w(f) given by 


wif) = 


Suif) 


Lkez Su{f-k/To) 


Moreover, the error in this estimation is 


mmse 


mu) = fjuif)df-J^^ Sw{f)df, 


(36) 


(37) 


where 


Swif) = Y \W{f-k/To)\^Su{f-k/To). (38) 

kel 

We conclude from the above that Dg{R), and therefore D(y|p(/s,R), is obtained by solving a source coding 
problem for an information source with a PAM signal structure, illustrated in Eigure |4] Since Example |2] implies 
that such a signal is CS with period 7^ = we can apply Theorem |2] in order to evaluate this DRE. By doing 
so, we obtain the following general result: 
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w{t) 



Fig. 4; Minimal distortion in Figure 0 is obtained by PAM followed by an optimal source code for the output of 
this modulation. 


Proposition 4 (DRF of PAM-modulated signals). Let Xpam{') be defined by 


XpAuit) = Y,U{nTo)p{t-nTo), t € [ 

nGlj 


(39) 


where U{■) is a Gaussian stationary process PSD Su{f) and p(t) is an analog deterministic signal with 

/”oo \p{f)\^ < °° Fourier transform P{f). Assume moreover, that the covariance function Rxp^^{t,T) ofXpAmi') 

is Lipschitz continuous in its second argument. The distortion-rate function ofXpAui’) is given by 


D{e) = -J^^mm{s{f),e}df 
m = ^ iog+ [sif)/e] df, 


where 


s{f) = E \P{f-k/TQfSu{f-k/T^). 


(40a) 

(40b) 

(41) 


Proof: See Appendix [D] ■ 

Proposition |4] applied to the process U{-) implies that its DRF D^iR) is given by waterfilling over the function 

J{f)=Y.\^[f-kfstSu{f-kf,). 

kez 

As a result, we obtain from (1351) and (l40l) the following expression for the minimal distortion in the combined 
sampling and source coding problem: 


^£/|(7(/i>-^) = mmse(t/|t7) + ^ mm{J{f),9}df, 

To 


where 


Rie) = - 1 


^ log+[7(/)/0]uf/. 


(42a) 


(42b) 


^Although we only use the value of U(t) at t € ZTq, it is convenient to treat U{-) as continuous-time source so that the expressions emerging 
have only continuous-time spectrum. 
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B. Information Content of Signals with PAM Structure 

PropositionlUprovides a general closed-form expression for the DRF of Gaussian processes with a PAM structure. 
In this subsection we use this expression to study the effect of the PAM of (l3^ on the distortion-rate curve of 
the signal Xpam{') at the output of the modulator. Assuming that two processes have the same energy over time, 
the process with lower DRF can be described by fewer bits per second to the same distortion level. It is therefore 
intuitive to think about the DRF as a measure of the information content of the procesj^ 

If we assume that the source for the symbols in the PAM is a Gaussian stationary process U{-), the output of 
the PAM of (IWt can be seen as a non-ideal reconstruction of t/(-) from its uniform samples using pulses of shape 
p{t), as illustrated in Figure |6] Since the randomness in Xpam{') is only due to U{-), we expect Xpam{') to have 
a smaller information content than U{-). In addition, we expect the information content of X<t>( ) to increase with 
the sampling rate 1 /Tq, and reach a saturation as this sampling rate exceeds the Nyquist rate of t/(-). Indeed, when 
l/7o is higher than the Nyquist rate of U{-), the support of Sij{f) is contained within ^“57^’ 2 ^)' case, 

expression (l40l i implies that the DRF of Xpam{') is obtained by waterhlling over the function 

5(/) = |P(/)|"%(/). (43) 

That is, the effect of the modulation in super-Nyquist sampling is identical to the effect of a linear hlter with 
frequency response P{f) applied to t/(). This filtering can be understood as a linear transformation of the coordinates 
||2^ Ch. 22] represented by the frequency components. Assuming that P{f) does not change the support of (|4^ 
(that is, the change in ‘coordinates’ is invertible), the process U{-) can be recovered from Xpam{ ) with zero mean- 
square etTor. When the sampling frequency I/Tq goes below the Nyquist rate of t/(-), perfect recovery of Uf) 
is in general not possible. Intuitively, in this case we expect Xpam{-) to contains less information than U{-), and 
hence we should be able to describe it under the same normalized distortion level as t/( ) using fewer bits per time 
unit. A quantitative evaluation of this effect of the sampling rate is given in Figure |6] where the DRF of Xpam{') 
is compared to the DRF of U{-) for three sub-Nyquist sampling rates. Examples for the realization of Xpam{-) and 
U{-) using sub- and super- Nyquist sampling rates are given in Figure |5] 

C. Amplitude Modulation with Random Phase 

In this section we turn back to the two processes discussed in the introduction as our motivating examples and 
evaluate their DRFs using Theorem |2] 

Consider the process X<i>(-) obtained by modulating a stationary Gaussian process Uf) by a cosine-wave of 
frequency /o and a random phase <I> uniform over [0,2;r), as dehned in ([T]). It is an elementary exercise HI Ex. 
8.18] to show that the process A<i>(-) is stationary with PSD 

s^{f) = \su{f-h) + \su{f+h). (44) 

^This notion is made precise by the notion of e-entropy (26), Ell 
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Fig. 5: Two realizations over time of the PAM process Xpam{') (blue) and the baseband process U{-) (dashed) with 
the PSD and pulse shape given in Figure |6l corresponding to sub-Nyquist (left) and super-Nyquist (right) sampling 
rates. Figure |6] below shows that for the same target distortion, the PAM-modulated process on the left is easier to 
describe than the PAM-modulated process on the right. 



Fig. 6: The DRF of the PAM signal (fTSl l for three values of sampling rate 1 /Tq compared to the Nyquist rate W of 
U{-). The DRF of the baseband stationary signal U{-) (assuming an energy preserving modulation) is given by the 
dashed curve. The PSD of U{-) and the shape of the pulse p{t) are given in the small frames. This figure shows 
that the information content of in the PAM process decreases with the sampling rate. 


From Is] Thm. 4.6.5], an upper bound on the DRF of X<j,(f), denoted by Dx^{R), is obtained by the DRF of a 
Gaussian process with the same PSD S^{f) through the reverse-waterfilling (|22|. Flowever, it seems that Dx^{R) 
cannot be determined solely from the second order statistics of Xt,( ). 

The main obstacle in deriving Dx^{R) is the random phase of which makes the process non-Gaussian and 
non-ergodic. This random phase can be handled using an asynchronous block code ||7] Ch. 11.6], i.e. by adding a 
short prefix consisting of a source synchronization word to each block. Indeed, the following proposition follows 
directly from the proof of Theorem 11.6.1 in [|7]: 
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Proposition 5. For any (p S [0,27r) (deterministic), the DRF of the process 2i’<i>(’) coincides with the DRF of the 
process 

X^{t) = \/2U{t)cos{2Kfot + (p), f G R. (45) 

It was noted in Example [T] above that is CS with the SCO function (fTSl) . It follows that is given by 

the DRF of the Gaussian CS process X^{-), generated by modulating the stationary Gaussian process U{-) using a 
deterministic cosine wave. Note that regardless of the carrier frequency /o, the baseband process U{-) can always 
be recovered from X,p{-), and that the s/2 factor implies that the modulation preserves energy. These two facts are 
not enough to guarantee equality between the DRFs of the processes, since the modulation may lead to a ‘change in 
coordinates’ in the spectrum, in analogy with (l43T l and Il28l Ch. 22]. In the following proposition we use Theorem|2] 
to show that this equality indeed holds as long as /o is bigger than twice the bandwidth of Sij{f). 

Proposition 6. Let U{-) be a Gaussian stationary process bandlimited to (—/b,/b)- Let fo > 2/b. The DRF of the 
process 

X^{t) = V2U{t) cos(27r/of+ (p), f G K, 
equals the DRF of the stationary Gaussian process Lf (■). 

Proof: See Appendix]^ ■ 

Proposition |7] asserts that the process X^{-) with AM signal structure suffers the same minimal distortion as the 
baseband process U (•) upon the encoding of each of them at rate R, and provided the latter is narrowband. Figure [T] 
shows that the above equality does not necessarily hold when U{-) is not narrowband. Propositions |5] and |6] leads 
to the following conclusion: 

Corollary 7. Let U(-) be a Gaussian stationary process bandlimited to (—/bj/b)- Assume that <I> is uniformly 
distributed over (0,2Tt) and fo > 2/b. The distortion rate function of the stationary process 

X^(t) = V2U{t)cos{2nfot+<P), f G M, 
equals the DRF of the baseband process !/(•). 

It is interesting to note that the DRF of a Gaussian process with the same PSD as the stationary process 
is strictly bigger than the DRF of the baseband process U{-), and therefore provides an upper bound to Dxj,(R). 
This upper bound is illustrated in Fig. 0 


VI. Conclusions 

We derived an expression for the distortion-rate function (DRF) of a class of Gaussian processes with periodically 
time-varying statistics, known as cyclostationary (CS) processes. This DRF is computed by reverse waterfilling 
over eigenvalues of a spectral density matrix associated with the polyphase components in the decomposition of the 
source. Unlike other general expression for the DRF of Gaussian processes that use orthogonal basis expansion over 
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Fig. 7; The DRF of the processes X(p{-) of (|45] | (blue), the DRF of the baseband process U{-) (dashed), and the 
lower bound of Proposition [3 The PSD Su{f) is taken to be the pulse given in the small frame. Proposition |6] 
implies that Dij{R) and Dx^{R) coincides for > 2/^. Also shown is the DRF of the Gaussian stationary process 
with PSD S^{f) (dotted), which gives an upper bound to Dx^{R). 


increasing but finite time intervals, the expression we derive exploits the CS of the process by orthogonalizing the 
polyphase components. Since these components are defined over the entire time horizon, the resulting expression 
can be expressed in terms of the spectrum of the process. In the continuous-time counterpart the solution is given 
in terms of a limit over functions of these eigenvalues. 

While we leave open the possibility whether there exists a closed form solution to the above limit in general, 
we have evaluated this limit in two special cases: a Gaussian CS processes with a PAM signal structure, and a 
Gaussian CS process with an amplitude modulation signal structure. As a result, we obtained the DRF of the 
processes obtained by these two important modulation schemes in terms of the power spectral density of the 
baseband stationary processes. We have also used the DRF result for a process with a PAM structure to derive the 
DRF of a process under combined sampling and source coding. 

In addition to an expression for the DRF of CS processes, we have derived a lower bound on this DRF obtained 
by averaging the minimal distortion attained in encoding each of the polyphase components over a single period. 
This bound is tight when high correlation among these components is present. 


20 









Appendix A 


In this Appendix we provide a proof of Theorem [T] Consider the vector valued process X^[-] dehned in (fT^ . 
The rate-distortion function of X^[-] is given by (l24l i: 

1 M. f-OO 

= (46a) 

m= \ 

1 A/ poo 

= T E / log^ /®] (46b) 

where 0 < Ai ^ < ... < Am ) are the eigenvalues of the spectral density matrix Sxm ) obtained by 
taking the Fourier transform of covariance matrix Rx[k] =E [X''^[n + k](X''^[n])^] entry-wise. The (m,r)* entry of 
SxM is given by (fTB : 

(Sx-(«“♦)),„=s;'''C’“*) 

= (47) 

^ k=0 ^ 

\V is left to show that the DRF of X^[-] coincides with the DRF of X[ ]. By the source coding theorem for AMS 
processes |j7] Thm. 11.4.1] it is enough to show that the operational block coding distortion-rate function (Q Ch. 
11.2]) of both processes is identical. Indeed, any N block codebook for X^[-] is an MN block codebook for X[-] 
which achieves the same quadratic distortion averaged over the block. However, since X''^[-] is stationary, by Cl 
Lemma. 11.2.3] we know that any distortion above the DRF of X'^[-] is attained for large enough N. This implies 
that the same is true for X[-]. 


Appendix B 

In this Appendix we prove Theorem |2] Given a Gaussian cyclostationary process X(-) with period 7b > 0, we 
dehne the discrete-time process X[-] obtained by uniformly sampling X(-) at intervals Tq/M, i.e. 

X[n] =X(n7b/M), n e Z. (48) 

The autocorrelation function of X[-] satisfies 

Rx[n+M,k]=W.[X[n+M + k]X[n+M]] 

= E [X(n7b/M + 7b -f kT^lM)X[nT^lM -f 7b)] 

= Rx{nTQ/M -f T(,,kT(,/M + Tq) 

^Rx{nTo/M,kTo/M) 

^Rx[n,k], 

which means that X[-] is a discrete-time Gaussian cyclostationary process with period M. The TPSD of X[-] is given 
by 
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This means that the PSD of the m* PC of Xf-l is 


S^ie 


1 , , . 

= s T 

n=0 


1 n ^wTq/m ((p—Mk — n 


- T’ E E 

-'0 n=0 keZ 


To 


— Y' ^ 

T„ ^ \ To I 


To 


By applying Theorem[T]to X[-], we obtain an expression for the DRF of X[-] as a function of M: 

2 M 


\ ri 

T>m{0m) = / m\n[Xm{e^^'^) ,dM]d<p 

1 M A 

^ E / . log+ [^n /0m] dP, 

where Xi < ■ ■ ■ < Xm are the eigenvalues of the matrix with (m+ l,r+ 1)* entry 


= tzY 

n=0 ^ 

j M-1 


- ^ E E ‘^x 

■'0 n=0 kez 


rT^/M ( p-n-kM '\^ ^2Kiim-r)i^ 


To 


= Yy 5''^o/M f 0_ i\ ^2ni(m-r)^ 

To h. "" \ To 


' /6Z 


(49a) 

(49b) 

(50) 


In order to express the code-rate in bits per time unit, we multiply the number of bits per sample R by the sampling 
rate M/Tq. This shows that the DRF of X[-], as measured in bits pertime unit R, is given by (|27] |. 


In order to complete the proof we rely on the following lemma: 

Lemma 8. Let X(-) be as in Theorem^ and let X[-] be its uniformly sampled version at rate M/To os in (I481 l. 
Denote the DRF at rate R bits per time unit of the two processes by D(R) and D{R), respectively. Then 

hm D{R)=D{R). 

M ->00 

The rest of the appendix is devoted to the proof of Lemma [8] 

Throughout the next steps it is convenient to use the covariance kernels K{t,s) = Rx{s,t — s) and K[n,k] = 
Rx[n,k — n]. For M £ N, define 

K{t,s) =K{[tM/To\ To/M, [sM/To\ To/M). 

For any fixed T >0, the kernel K{t,s) defines an Hermitian positive compact operator E9\ on the space of square 
integrable functions over [—T,T]. The eigenvalues of this operator are given by the Fredholm integral equation 

Xj,{t) = ^ f_^K{t,s)fi{s)ds, -T<t<T, (51) 
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where it can be shown that there are at most MT /Tq non-zero eigenvalues {A;} that satisfy ( fSTI ). We define the 
function Dt{R) by the following parametric expression: 

Dt{0) = f]min|A;,0| 
i=i '' ^ 

/- X (52) 

(the eigenvalues in (l52]) are implicitly depend on T). Note that 

OO I fT ^ 1 ^ 

= — K{t,t)dt = — K{nTo/M,nTo/M), (53) 

2T J-t 2T 

where N = MT/Tq. Expression (l5^ converges to 

as M goes to infinity due to our assumption that R{t,x) is Riemann integrable and therefore so is K{t,s). Since 
we are interested in the asymptotic of large M, we can assume that (l53T l is bounded. This implies that Dt{R) is 
bounded. 

We would like to claim that the eigenvalues {A;} approximate the eigenvalues {A/}. We have the following 
lemma: 


Lemma 9. Let {A/} and {A;} be the eigenvalues in the Fredholm integral equation ofK(t,s) and K(t,s), respectively. 
Assume that these eigenvalues are numbered in a descending order. Then 


A/ — A; 


<4CTo/M, 1 = 1,2,.... 


(54) 


Proof of Lemma^ Approximations of the kind (l54l l can be obtained by Weyl’s inequalities for singular values 
of operators defined by self-adjoint kernels ll^ . In our case it suffices to use the following result lISTl Cor. 1”]: 


A/ — A/ 

< 2 sup 

>1 

1 

>1 


t,se[-T,T] 



1 = 1 , 2 ,.... 


(55) 


The assumption that Rx{t,T) is Lipschitz continuous in T implies that there exists a constant C > 0 such that for 
any tif 2 ,s G M, 

\Kiti,s)-K{t 2 ,s)\ = \Rx{s,ti - s) - Rxis,t 2 - s) \ < C|fi -f 2 |. 


We therefore conclude that Kx{t,s) is Lipschitz continuous in both of its arguments from symmetry. Lipschitz 
continuity of K{t,s) implies 


\K{tl,Si)-K{t2,S2)\ 

< |/r(fi,Si) — fir(fi,S2)| + K{tl,S2) — K{t2,S2) 


< C|si -S 2 I +C|fi -f2| . 
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As a result, dSD leads to 


A; — A/ 


< 2 sup 

t,s 


K{t,s)-K{t,s) 


= 2 sup |/:(f,s)-/:(LfM/rojro/M,LsM/7’ojro/M)| 
t,se[-T,T] 

<2C{\t - [tM/To\To/M\ + \t - [sM/To\To/M\) 


< ACTq/M, 


which proves Lemma 0 


The significance of Lemma |9] is that the eigenvalues of the kernel K{t,s) used in the expression for the DRF of 
A'(-) can be approximated by the eigenvalues of K{t,s), where the error in each of these approximations converge, 
uniformly in T, to zero as M increases. Since only a finite number of eigenvalues participate in (|2TI) and since both 
Dt{R) and Dt{R) are bounded continuous functions of their eigenvalues, we conclude that Dt{R) converges to 
Dt{R) uniformly in T. 

Now let e > 0 and fix Mq large enough such that for all M > Mq and for all T 

\DT{R)-bT{R)\<e. (56) 

Recall that in addition to (|2^ . the DRF of A[-], denoted here as b{R), can also be obtained as the limit in N of 
the expression 

DNid) = f]min{A,, 0 } 

i=i 

^( 0 )=^f;iog+ (A// 0 ), 

where Ai, A 2 ,... are the eigenvalues in the KL expansion of X over n = —N, 

hfi[n] = :r^ t Kxhmk]. l = h...,N, (57) 

(there are actually at most 2N + 1 distinct non-zero eigenvalues that satisfies dSTl)'). Letting 7]v = TqM/N and 
//(f) = fi {\t/T q\M) (ISTT i can also be written as 

hfi[n] = r Kx{nTo/M,s)f, [[s/To\M]ds, I = 1,2,..., 

J-Tn 

= / K{t,s)fi{s)ds, -Tn <t <Tn. 

J —Tn 

From the uniqueness of the KL expansion, we obtain that for any N, the eigenvalues of K{t,s) over Tv = TqM/N 
are given by the eigenvalues of K[n,k] over —N,... ,N. We conclude that 

DNiR)=DT^{R), (58) 

where R = RTq/M. Now take N large enough such that 

|D/v(R)-D(R)| <e. 
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and 


\DTAR)-D{R)\<e. 


For all M ^ A/q we have 


\DiR)-D{R)\ = \D{R)-Dt,{R)+Dt,{R) +DtAR) 
-Dt^{R) +DN{R)-DNiR)-DiR)\ 
<\D{R)-DT,iR)\ 

+ \DtAR)-DtAR)\ 

+ \DT^iR)-DN{R)\ 

+ \DN{R)-D{R)\<3e, 


(59) 

(60) 
( 61 ) 
(62) 


where the last transition is because: ( |59] ) and ( |62] ) are smaller than e by the choice of N, (|60] | is smaller than e 
from (l5^ . and (IhTI) equals zero from (l58l l. 


Appendix C 


In this Appendix we provide a proof of Proposition [3] We use the process [•] defined in the proof of Theorem|2] 
as the uniform sampled version of X(-) at rate To/M. From Proposition [T] we conclude that the DRF of X[-] satisfies 

M-l ' 


1 M—L 1 

Dx{R)>- ^ / ,mini 

^ m=0 ■’-J 1 ^0 ;gZ 




To/M ( 

To 


, Oyfi / dij) j 


(63) 


where for all in — 0,... ,M— 1, 9^ is determined by 


R 


-T 


log^ 


— V ( — _ - 

[ To 


/Qm 


d(j). 


Denote t = inTo/M. As M approaches infinity, the RHS of (l63t converges to an integral with respect to t over the 
interval (0,7o), which implies 


and 



(64) 


(65) 


where we denoted Qt — To9„,. In order to go from R to R we multiply dhST l by M/Tq, so that (l64l i and (l65T l lead to 
(l3^ . The fact that the function D{R) converges to D{R) as M goes to infinity follows from the proof of Theorem. |2] 
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Appendix D 


In this Appendix we provide a proof of Proposition |4] The entries of the matrix in Theorem |2] are 

obtained by using the CPSD of the PAM process (fTTl) in d^ . For all M G N, this leads to 


^m+l,r+l 
1 


T? 


E 

0 k<£l L 




7b 


7 b 


Y^p* 


(j) —n — k 

7b 


OviP’’ 


Lv p( Su ( 


= ^Yp 


'0 A:GZ 
X Yp 


To 


To 


^ _ i] e-2®'(b-05& 

7 b 


( 66 ) 

(67) 


The expression (l67l l consist of the product of a term depending only on m and a term depending only on r. We 
conclude that the matrix S can be written as the outer product of two M dimensional vector, and thus it is 

of rank one. The single non-zero eigenvalue Am of S is given by the trace of the matrix, which, by 

the orthogonality of the functions in (l6^ . is evaluated as 


AM(e2"'^) 


M ^ 

y2 

-'o kez 




( 68 ) 


We now use (l68l) in dZTl l. In order to obtain (l40l) . we change the integration variable from (j) to / = 0/7b and the 
water-level parameter 9 to TqO/M. Note that the final expression is independent of M, so the limit in ( |29] | is already 
given by this expression. 


Appendix E 

In this Appendix we provide a proof of Proposition |6] Since Su (/) is compactly supported, the covariance 
function Rui'^) =^U{t + T)U{t) is an analytic function and therefore Lipshitz continuous. Lipschitz continuity of 
Rui'^) implies Lipschitz continuity of Rx{t,T) in its second argument and therefore Theorem |2] applies: The DRF 
of the Gaussian CS process X^{-) with period 7b = is obtained by using Theorem |2] with the SCO (flSl) . For 
all M G N and m, r = 0,... ,M — 1 we have, 

S„,+u+i = 

y E ^£2 (/o (0 - 1 )) (l -f 

+ ^Y^uiM^-k+l))(l+ 

^ kei ^ 
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Under the assumption that /o > Ifg we have that for all 0 G (—5), S(/ (/o(^ —kiL 1)) = 0 for all k ^ ±1. This 
leads to 


S„+i.r+i 


+ %(/o0) 


/o(l 


-47rz> 






= 2foSu{M)e'^''‘^^ COS cos ■ 

From dTOl l we conclude that the matrix S ) can be written as 

S = 2foSu{M)SM , 


(69) 


(70) 


where Sm G is given by 

(^1,^>2^'^/"C0S ,. . . COS ) • 

This means that S is a matrix of rank one, and its single non-zero eigenvalue is given by its trace: 

M-l 

=2/o5c;(/o0) ^ cos^27Zm/M)=MfoSuiM). 

m=0 

We use this in (l28T l: 

f M , 1 

RM{e) = \ iog+ [1™ 

= f /_" iog+ [MfoSuiM)/e]d^ 

= y^Jog+[Su{f)/{0/M)]df, (71) 

and 

£>m( 0) = ^J\ mm{MfoSuiM),9}d^ 

= fof mm{Su{M),9/M}d^ 

J — 00 

/ oo 

^mm{Suif),d/M}df. (72) 

From dZB and (I72I 1 we conclude that for every M, the parametric expression of D as a function of R is identical 
to the DRF of the stationary process U{-) given by (l22ll . 
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